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Abstract —Protocols for tasks such as authentication, elec¬ 
tronic voting, and secure multiparty computation ensure desir¬ 
able security properties if agents follow their prescribed pro¬ 
grams. However, if some agents deviate from their prescribed 
programs and a security property is violated, it is Important 
to hold agents accountable by determining which deviations 
actually caused the violation. Motivated by these applications, 
we Initiate a formal study of program actions as actual causes. 
Speciflcally, we define in an interacting program model what 
it means for a set of program actions to be an actual cause 
of a violation. We present a sound technique for establishing 
program actions as actual causes. We demonstrate the value of 
this formalism in two ways. First, we prove that violations of a 
specific class of safety properties always have an actual cause. 
Thus, our definition applies to relevant security properties. 
Second, we provide a cause analysis of a representative protocol 
designed to address weaknesses in the current public key 
certification infrastructure. 

Keywords-Security Protocols, Accountability, Audit, Causa¬ 
tion 

I. Introduction 

Ensuring accountability for security violations is essential 
in a wide range of settings. For example, protocols for 
authentication and key exchange Cl, electronic voting m, 
auctions a, and secure multiparty computation (in the semi- 
honest model) El ensure desirable security properties if 
protocol parties follow their prescribed programs. However, 
if they deviate from their prescribed programs and a security 
property is violated, determining which agents should be 
held accountable and appropriately punished is important 
to deter agents from committing future violations. Indeed 
the importance of accountability in information systems has 
been recognized in prior work IS), ||6l, ||71, a, a, IfTOl . 
CD. Our thesis is that actual causation (i.e., identifying 
which agents’ actions caused a specific violation) is a useful 
building block for accountability in decentralized multi¬ 
agent systems, including but not limited to security protocols 
and ceremonies na. 

Causation has been of interest to philosophers and ideas 


from philosophical literature have been introduced into 
computer science by the seminal work of Halpern and 
Pearl IH, IH, El. In particular, counterfactual reasoning 
is appealing as a basis for study of causation. Much of 
the definitional activity has centered around the question of 
what it means for event c to be an actual cause of event 
e. An answer to this question is useful to arrive at causal 
judgments for specific scenarios such as “John’s smoking 
causes John’s cancer” rather than general inferences such as 
“smoking causes cancer” (The latter form of judgments are 
studied in the related topic of type causation csi). Notably, 
Hume El identified actual causation with counterfactual 
dependence—the idea that c is an actual cause of e if had 
c not occurred then e would not have occurred. While this 
simple idea does not work if there are independent causes, 
the counterfactual interpretation of actual causation has been 
developed further and formalized in a number of influential 
works (see, for example, El, El, El, El, EOl, El)- 
Even though applications of counterfactual causal analysis 
are starting to emerge in the fields of AI, model-checking, 
and programming languages, causation has not yet been 
studied in connection with security protocols and violations 
thereof. On the other hand, causal analysis seems to be an 
intuitive building block for answering some very natural 
questions that have direct relevance to accountability such as 
(i) why a particular violation occurred, (ii) what component 
in the protocol is blameworthy for the violation and (iii) 
how the protocol could have been designed differently to 
preempt violations of this sort. Answering these questions 
requires an in-depth study of, respectively, explanations, 
blame-assignment, and protocol design, which are interest¬ 
ing problems in their own right, but are not the explicit focus 
of this paper. Instead, we focus on a formal definition of 
causation that we believe formal studies of these problems 
will need. Roughly speaking, explanations can be used to 
provide an account of the violation, blame assignment can 
be used to hold agents accountable for the violation, and 
protocol design informed by these would lead to protocols 


with better accountability guarantees. We further elaborate 
on explanations and blame-assignment in Section [V] 

Formalizing actual causes as a building block for ac¬ 
countability in decentralized multi-agent systems raises new 
conceptual and technical challenges beyond those addressed 
in the literature on events as actual causes. In particular, 
prior work does not account for the program dynamics that 
arise in such settings. Let us consider a simple protocol 
example. In the movie Flight im, a pilot drinks and snorts 
cocaine before flying a commercial plane, and the plane goes 
into a locked dive in mid-flight. While the pilot’s behavior 
is found to be deviant in this case—he does not follow 
the prescribed protocol (program) for pilots—it is found 
to not be an actual cause of the plane’s dive. The actual 
cause was a deviant behavior by the maintenance staff— 
they did not replace a mechanical component that should 
have been replaced. Ideally, the maintenance staff should 
have inspected the plane prior to take-off according to their 
prescribed protocol. 

This example is useful to illustrate several key ideas that 
influence the formal development in this paper. First, it 
illustrates the importance of capturing the actual interactions 
among agents in a decentralized multi-agent system with 
non-deterministic execution semantics. The events in the 
movie could have unfolded in a different order but it is 
clear that the actual cause determination needs to be done 
based on the sequence of events that happened in reality. 
For example, had the maintenance staff replaced the faulty 
component before the take-off the plane would not have 
gone into a dive. Second, the example motivates us to hold 
accountable agents who exercise their choice to execute a 
deviant program that actually caused a violation. The main¬ 
tenance staff had the choice to replace the faulty component 
or not where the task of replacing the component could 
consist of multiple steps. It is important to identify which 
of those steps were crucial for the occurrence of the dive. 
Thus, we focus on formalizing program actions executed in 
sequence (by agents) as actual causes of violations rather 
than individual, independent events as formalized in prior 
work. Finally, the example highlights the difference between 
deviance and actual causes—a difference also noted in prior 
work on actual causation. This difference is important from 
the standpoint of accountability. In particular, the punish¬ 
ment for deviating from the prescribed protocol could be 
suspension or license revocation whereas the punishment for 
actually causing a plane crash in which people died could be 
significantly higher (e.g., imprisonment for manslaughter). 
The first and second ideas, reflecting our program-based 
treatment, are the most significant points of difference from 
prior work on actual causation lfT4ll . fT2\ while the third 
idea is a significant point of difference from prior work in 
accountability ||23l, ll24ll . ifTTll . 111. 

The central contribution of this paper is a formal def¬ 
inition of program actions as actual causes. Specifically, 


we define what it means for a set of program actions to 
be an actual cause of a violation. The definition considers 
a set of interacting programs whose concurrent execution, 
as recorded in a log, violates a trace property. It identifies 
a subset of actions (program steps) of these programs as 
an actual cause of the violation. The definition applies in 
two phases. The first phase identifies what we call Lamport 
causes. A Lamport cause is a minimal prefix of the log of 
a violating trace that can account for the violation. In the 
second phase, we refine the actions on this log by removing 
the actions which are merely progress enablers and obtain 
actual action causes. The former contribute only indirectly 
to the cause by enabling the actual action causes to make 
progress; the exact values returned by progress enabling 
actions are irrelevant. 

We demonstrate the value of this formalism in two ways. 
First, we prove that violations of a precisely defined class 
of safety properties always have an actual cause. Thus, our 
definition applies to relevant security properties. Second, 
we provide a cause analysis of a representative protocol 
designed to address weaknesses in the current public key 
certification infrastructure. Moreover, our example illustrates 
that our definition cleanly handles the separation between 
joint and independent causes -a recognized challenge for 
actual cause definitions m, m, 03. 

In addition, we discuss how this formalism can serve as a 
building block for causal explanations and exoneration (i.e., 
soundly identifying agents who should not be blamed for 
a violation). We leave the technical development of these 
concepts for future work. 

The rest of the paper is organized as follows. Section [H] 
describes a representative example which we use throughout 
the paper to explain important concepts. Section |III] gives 
formal definitions for program actions as actual causes of 
security violations. We apply the causal analysis to the 
running example in Section HV] We discuss the use of our 
causal analysis techniques for providing explanations and 
assigning blame in Section |V] We survey additional related 
work in Section [Vl] and conclude in Section IVIII 

IT Motivating example 

In this section we describe an example protocol designed 
to increase accountability in the current public key infras¬ 
tructure. We use the protocol later to illustrate key concepts 
in defining causality. 

Security protocol: Consider an authentication protocol 
in which a user (Userl) authenticates to a server (Serverl) 
using a pre-shared password over an adversarial network. 
Userl sends its user-id to Serverl and obtains a public 
key signed by Serverl. However, Userl would need inputs 
from additional sources when Serverl sends its public key 
for the first time in a protocol session to verify that the 
key is indeed bound to Serverl’s identity. In particular, 
Userl can verify the key by contacting multiple notaries 
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in the spirit of Perspectives dsn. For simplicity, we assume 
Userl verifies Serverl’s public key with three authorized 
notaries—Notaryl, Notary2, NotaryS—and accepts the key 
if and only if the majority of the notaries say that the key is 
legitimate. To illustrate some of our ideas, we also consider 
a parallel protocol where two parties (User2 and User3) 
communicate with each other. 

We assume that the prescribed programs for Serverl, 
Userl, Notaryl, Notary2, NotaryS, User2 and User3 impose 
the following requirements on their behavior; (i) Serverl 
stores Userl’s password in a hashed form in a secure 
private memory location, (ii) Userl requests access to the 
account by sending an encryption of the password (along 
with its identity and a timestamp) to Serverl after verifying 
Serverl’s public key with a majority of the notaries, (iii) The 
notaries retrieve the key from their databases and attest the 
key correctly, (iv) Serverl decrypts and computes the hashed 
value of the password, (v) Serverl matches the computed 
hash value with the previously stored value in the memory 
location when the account was first created; if the two hash 
values match, then Serverl grants access to the account to 
Userl. (vi) In parallel, User2 generates and sends a nonce to 
User3. (vii) User3 generates a nonce and responds to User2. 

Security property: The prescribed programs in our 
example aim to achieve the property that only the user who 
created the account and password (in this case, Userl) gains 
access to the account. 

Compromised Notaries Attack: We describe an attack 
scenario and use it to illustrate nuances in formalizing pro¬ 
gram actions as actual causes. Userl executes its prescribed 
program. Userl sends an access request to Serverl. An 
Adversary intercepts the message and sends a public key to 
Userl pretending to be Serverl. Userl checks with Notaryl, 
Notary2 and Notary3 who falsely verify Adversary’s public 
key to be Serverl’s key. Consequently, Userl sends the 
password to Adversary. Adversary then initiates a protocol 
with Serverl and gains access to Userl’s account. In parallel, 
User2 sends a request to Serverl and receives a response 
from Serverl. Following this interaction, User2 forwards the 
message to User3. We assume that the actions of the parties 
are recorded on a log, say 1. Note that this log contains 
a violation of the security property described above since 
Adversary gains access to an account owned by Userl. 

First, our definition finds program actions as causes of 
violations. At a high-level, as mentioned in the introduction, 
our definition applies in two phases. The first phase (Sec¬ 
tion Uni Definition fT2li identifies a minimal prefix (Phase 1, 
minimality) of the log that can account for the violation 
i.e. we consider all scenarios where the sequence of actions 
execute in the same order as on the log, and test whether it 
suffices to recreate the violation in the absence of all other 
actions (Phase 1, sufficiency). In our example, this first phase 
will output a minimal prefix of log I above. In this case, the 
minimal prefix will not contain interactions between User2 


and User3 after Serverl has granted access to the Adversary 
(the remaining prefix will still contain a violation). 

Second, a nuance in defining the notion of sufficiency 
(Phase 1, Definition fT2l) is to constrain the interactions which 
are a part of the actual cause set in a manner that is consistent 
with the interaction recorded on the log. This constraint 
on interactions is quite subtle to define and depends on 
how strong a coupling we find appropriate between the 
log and possible counterfactual traces in sufficiency: if the 
constraint is too weak then the violation does not reappear 
in all sequences, thus missing certain causes; if it is too 
strong it leads to counter-intuitive cause determinations. For 
example, a weak notion of consistency is to require that each 
program locally execute the same prefix in sufficiency as it 
does on the log i.e. consistency w.r.t. program actions for 
individual programs. This notion does not work because for 
some violations to occur the order of interactions on the log 
among programs is important. A notion that is too strong is 
to require matching of the total order of execution of all 
actions across all programs. We present a formal notion of 
consistency by comparing log projections (Section riII-Bt that 
balance these competing concerns. 

Third, note that while Phase 1 captures a minimal prefix 
of the log sufficient for the violation, it might be possible to 
remove actions from this prefix which are merely required 
for a program execution to progress. For instance note that 
while all three notaries’ actions are required for Userl to 
progress (otherwise it would be stuck waiting to receive a 
message) and the violation to occur, the actual message sent 
by one of the notaries is irrelevant since it does not affect 
the majority decision in this example. Thus, separating out 
actions which are progress enablers from those which pro¬ 
vide information that causes the violation is useful for fine¬ 
grained causal determination. This observation motivates the 
final piece (Phase 2) of our formal definition (Definition [141). 

Finally, notice that in this example Adversary, Notaryl, 
Notary2, Notary3, Serverl and User2 deviate from the 
protocol described above. However, the deviant programs are 
not sufficient for the violation to occur without the involve¬ 
ment of Userl, which is also a part of the causal set. We thus 
seek a notion of sufficiency in defining a set of programs 
as a joint actual cause for the violation. Joint causation is 
also significant in legal contexts ll26l . For instance, it is 
useful for holding liable a group of agents working together 
when none of them satisfy the cause criteria individually but 
together their actions are found to be a cause. The ability 
to distinguish between joint and independent (i.e., different 
sets of programs that independently caused the violation) 
causes is an important criterion that we want our definition 
to satisfy. In particular. Phase 2 of our definition helps 
identify independent causes. For instance, in our example, 
we get three different independent causes depending on 
which notary’s action is treated as a progress enabler. Our 
ultimate goal is to use the notion of actual cause as a building 
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block for accountability — the independent vs. joint cause 
distinction is significant when making deliberations about 
accountability and punishment for liable parties. We can 
use the result of our causal determinations to further remove 
deviants whose actions are required for the violation to occur 
but might not be blameworthy (Section IVT). 

III. Actual Cause Definition 

We present our language model in Section ITlI-AI auxiliary 
notions in Section lTlI-Bl properties of interest to our analysis 
in Section lTlI-CI and the formal definition of program actions 
as actual causes in Section UlI-DI 

A. Model 

We model programs in a simple concurrent language, 
which we call L. The language contains sequential expres¬ 
sions, e, that execute concurrently in threads and communi¬ 
cate with each other through send and recv commands. 
Terms, t, denote messages that may be passed through 
expressions or across threads. Variables x range over terms. 
An expression is a sequence of actions, a. An action may 
do one of the following: execute a primitive function on 
a term t (written or send or receive a message to 

another thread (written send(f) and recv{), respectively). 
We also include very primitive condition checking in the 
form of a s s e rt(t). 

Terms t ::= x \ ... 

Actions a ::= Q{t) \ send(f) | recv() 

Expressions e ::= t | {b:x = a);e 2 \ assert(f);e 

Each action a is labeled with a unique line number, 
written b. Line numbers help define traces later. We omit 
line numbers when they are irrelevant. Every action and 
expression in the language evaluates to a term and potentially 
has side-effects. The term returned by action a is bound to 
X in evaluating 62 in the expression {b : x = a); 62 - 

Eollowing standard models of protocols, send and recv 
are untargeted in the operational semantics: A message 
sent by a thread may be received by any thread. Targeted 
communication may be layered on this basic semantics using 
cryptography. Eor readability in examples, we provide an 
additional first argument to send and recv that specifies 
the intended target (the operational semantics ignore this 
intended target). Action send(f) always returns 0 to its 
continuation. 

Primitive functions C, model thread-local computation like 
arithmetic and cryptographic operations. Primitive functions 
can also read and update a thread-local state, which may 
model local databases, permission matrices, session infor¬ 
mation, etc. If the term t in assert(f) evaluates to a non- 
true value, then its containing thread gets stuck forever, else 
assert(f) has no effect. 

We abbreviate {b : x = a); x to b : a and {b : x = a); e 
to (6 : a);e when x is not free in e. As an example. 


the following expression receives a message, generates a 
nonce (through a primitive function new) and sends the 
concatenation of the received message and the nonce on 
the network to the intended recipient j (line numbers are 
omitted here). 

m = recv(); //receive message, bind to m 
n = new(); //generate nonce, bind to n 
send(j, (m, n)); //send (m, n) to j 

Eor the purpose of this paper, we limit attention to this 
simple expression language, without recursion or branching. 
Our definition of actual cause is general and applies to 
any formalism of (non-deterministic) interacting agents, but 
the auxiliary definitions of log projection and the function 
dummify introduced later must be modified. 

Operational Semantics: The language L’s operational 
semantics define how a collection of threads execute con¬ 
currently. Each thread T contains a unique thread identifier 
i (drawn from a universal set of such identifiers), the 
executing expression e, and a local store. A configuration 
C = Ti, ... ,Tn models the threads Ti,...,T„ executing 
concurrently. Our reduction relation is written C ^ C' and 
defined in the standard way by interleaving small steps of 
individual threads (the reduction relation is parametrized 
by a semantics of primitive functions Q. Importantly, each 
reduction can either be internal to a single thread or a 
synchronization of a send in one thread with a recv in 
another thread. 

We make the locus of a reduction explicit by annotating 
the reduction arrow with a label r. This is written C ^ C. 
A label is either the identifier of a thread i paired with a 
line number b, written {i, b) and representing an internal 
reduction of some Q{t) in thread i at line number b, or a tuple 
{{is,bs), {ir, hr)), representing a synchronization between a 
send at line number bg in thread ig with a recv at line 
number br in thread ir, or e indicating an unobservable 
reduction (of t or assert(f)) in some thread. Labels (i, b) 
are called local labels, labels {{is,bg), {ir,br)) are called 
synchronization labels and labels e are called silent labels. 

An initial configuration can be described by a triple 
{/, ,4., S), where / is a finite set of thread identifiers, 
A : I ^ Expressions and T, : I ^ Stores. This defines 
an initial configuration of |/| threads with identifiers in I, 
where thread i contains the expression A{i) and the store 
E(i). In the sequel, we identify the triple (/, A, S) with the 
configuration defined by it. We also use a configuration’s 
identifiers to refer to its threads. 

Definition 1 (Run): Given an initial configuration Cq = 
(/, 4., S), a run is a finite sequence of labeled reductions 
Co ^ Cl... ^ C„. 

A pre-trace is obtained by projecting only the stores from 
each configuration in a run. 

Definition 2 (Pre-trace): Let Cq Ci... C„ 

be a run and let be the store in configuration 
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Ci- Then, the pre-trace of the run is the sequence 

(_j ^o)j (fl, Si), . . . , {Vn, S„). 

If ri = e, then the ith step is an unobservable reduction 
in some thread and, additionally, Si_i = A trace is a 
pre-trace from which such e steps have been dropped. 

Definition 3 (Trace): The trace of the pre-trace 
(_, So), (ri, Si),..., (r„, S„) is the subsequence obtained 
by dropping all tuples of the form (e, S^). Traces are 
denoted with the letter t. 

B. Logs and their projections 

To define actual causation, we find it convenient to 
introduce the notion of a log and the log of a trace, which is 
just the sequence of non-silent labels on the trace. A log is 
a sequence of labels other than e. The letter I denotes logs. 

Definition 4 (Log): Given a trace t = 

(_, So),(ri,Si),...,(r„,S„), the log of the trace, 
log{t), is the sequence of ri,..., r^. (The trace t does not 
contain a label that equals e, so neither does log{t).) 

We need a few more straightforward definitions on logs 
in order to define actual causation. 

Definition 5 (Projection of a log): Given a log I and a 
thread identifier i, the projection of I to i, written l\i is the 
subsequence of all labels in / that mention i. Formally, 

•|z = • 

{{i,b) l)\, = (i^b) :: {l\^) 

{{i,b) l)\, = l\, if i^j 

(((*s; bg), (ir, br)) l)\i = {{is, bs), {ir, br)) (/|i) 

if is = j or v = * 

{{{is, bs), {ir, br)) l)\i = l\i 

if is i and i 

Definition 6 (Projected prefix): We call a log I' a pro¬ 
jected prefix of the log I, written V <p I, if for every thread 
identifier i, the sequence l'\i is a prefix of the sequence l\i. 

The definition of projected prefix allows the relative order 
of events in two different non-communicating threads to 
differ in / and I' but Lamport’s happens-before order of 
actions ll27ll in /' must be preserved in 1. Similar to projected 
prefix, we define projected sublog. 

Definition 7 (Projected sublog): We call a log V a pro¬ 
jected sublog of the log /, written I' Cp I, if for every 
thread identifier i, the sequence l'\i is a subsequence of the 
sequence l\i (i.e., dropping some labels from l\i results in 

I'U)- 

C. Properties of Interest 

A property is a set of (good) traces and violations are 
traces in the complement of the set. Our goal is to define 
the cause of a violation of a property. We are specifically 
interested in ascribing causes to violations of safety proper¬ 
ties ll^ because safety properties encompass many relevant 
security requirements. We recapitulate the definition of a 
safety property below. Briefly, a property is safety if it is 


fully characterized by a set of finite violating prefixes of 
traces. Let U denote the universe of all possible traces. 

Definition 8 (Safety property A property P (a set 

of traces) is a safety property, written Safety(P), if Vf ^ 
P. 3t' S U. {f is a prefix of t) A (Vf" G U. {f -t" ^ P)). 

As we explain soon, our causal analysis ascribes thread 
actions (or threads) as causes. One important requirement 
for such analysis is that the property be closed under 
reordering of actions in different threads if those actions 
are not related by Lamport’s happens-before relation IITTII . 
For properties that are not closed in this sense, the global 
order between actions in a race condition may be a cause of 
a violation. Whereas causal analysis of race conditions may 
be practically relevant in some situation, we limit attention 
only to properties that are closed in the sense described here. 
We call such properties reordering-closed or RC. 

Definition 9 (Reordering-equivalence): Two traces ti,t 2 
starting from the same initial configuration are called 
reordering-equivalent, written ti ^ t 2 if for each thread 
identifier i, log{ti)\i = log{t 2 )\i. Note that ~ is an equiv¬ 
alence relation on traces from a given initial configuration. 
Let [f]...., denote the equivalence class of t. 

Definition 10 (Reordering-closed property): A property 
P is called reordering-closed, written RC(P), if t € P 
implies C P. Note that RC(P) iff RC(-iP). 

D. Program Actions as Actual Causes 

In the sequel, let ipv denote the complement of a 
reordering-closed safety property of interest. (The subscript 
V stands for “violations”.) Consider a trace t starting from 
the initial configuration Cq = {I,A,T,). If f G (pv, then t 
violates the property -^pv- 

Definition 11 (Violation): A violation of the property 
^pv is a trace t G pv- 

Our definition of actual causation identifies a subset of 
actions in {,4(i) | i G /} as the cause of a violation t G 
Pv- The definition applies in two phases. The first phase 
identifies what we call Lamport causes. A Lamport cause 
is a minimal projected prefix of the log of a violating trace 
that can account for the violation. In the second phase, we 
refine the log by removing actions that are merely progress 
enablers', the remaining actions on the log are the actual 
action causes. The former contribute only indirectly to the 
cause by enabling the actual action causes to make progress; 
the exact values returned by progress enabling actions are 
irrelevant. 

The following definition, called Phase 1, determines 
Lamport causes. It works as follows. We first identify a 
projected prefix I of the log of a violating trace t as a 
potential candidate for a Lamport cause. We then check 
two conditions on 1. The sufficiency condition tests that 
the threads of the configuration, when executed at least up 
to the identified prefix, preserving all synchronizations in 
the prefix, suffice to recreate the violation. The minimality 
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condition tests that the identified Lamport cause contains no 
redundant actions. 

Definition 12 (Phase 1: Lamport Cause of Violation): 

Let t G ipy be a trace starting from Cq = {I, A, S) and / 
be a projected prefix of log{t), i.e., I <p log{t). We say 
that I is the Lamport cause of the violation t of (pv if the 
following hold; 

1) (Sufficiency) Let T be the set of traces starting from 
Cq whose logs contain I as a projected prefix, i.e., T = 
{t' I is a trace starting from Cq and I <p log{t')}. 
Then, every trace in T has the violation ipv, i-e., T C 
Pv (Because t G T, T is non-empty.) 

2) (Minimality) No proper prefix of / satisfies condition[T] 

At the end of Phase 1, we obtain one or more minimal pre¬ 
fixes I which contain program actions that are sufficient for 
the violation. These prefixes represent independent Lamport 
causes of the violation. In the Phase 2 definition below, we 
further identify a sublog ad of each I, such that the program 
actions in ad are actual causes and the actions in l\ad are 
progress enabling actions which only contribute towards the 
progress of actions in ad that cause the violation. In other 
words, the actions not considered in ad contain all labels 
whose actual returned values are irrelevant. 

Briefly, here’s how our Phase 2 definition works. We 
first pick a candidate projected sublog ad of /, where log 
I is a Lamport cause identified in Phase 1. We consider 
counterfactual traces obtained from initial configurations in 
which program actions omitted from ad are replaced by 
actions that do not have any effect other than enabling the 
program to progress (referred to as no-op). If a violation 
appears in all such counterfactual traces, then this sublog 
ad is a good candidate. Of all such good candidates, we 
choose those that are minimal. 

The key technical difficulty in writing this definition is 
replacing program actions omitted from ad with no-ops. 
We cannot simply erase any such action because the action 
is expected to return a term which is bound to a variable 
used in the action’s continuation. Hence, our approach is 
to substitute the variables binding the returns of no-op’ed 
actions with arbitrary (side-effect free) terms t. Formally, 
we assume a function f : I x LineNumbers —Terms that 
for line number b in thread i suggests a suitable term f{i,b) 
that must be returned if the action from line b in thread i is 
replaced with a no-op. In our cause definition we universally 
quantify over /, thus obtaining the effect of a no-op. For 
technical convenience, we define a syntactic transform called 
dummifyO that takes an initial configuration, the chosen 
sublog ad and the function /, and produces a new initial 
configuration obtained by erasing actions not in ad by terms 
obtained through /. 

Definition 13 (Dummifying transformation): Let 
{I, A, E) be a configuration and let ad be a log. Let 
f : I X LineNumbers —Terms. The dummifying transform 
dummify(/, ,4,, E, Od, /) is the initial configuration 


(/, X>, E), where for all i G I, D{i) is A{i) modified 
as follows: 

• If (6 : a: = send(f)); e appears in A{i) but (i, b) does 
not appear in ad, then replace {b : x = send(f));e 
with e[0/x] in A(^i). 

• If {b ■. X = a);e appears in A{i) but {i,b) does not 
appear in ad and a send(_), then replace {b : x = 
a);e with e[f{i,b)/x] in A{i). 

We now present our main definition of actual causes. 

Definition 14 (Phase 2: Actual Cause of Violation): Let 
t G Pv be a trace from the initial configuration (I, A, E) 
and let the log I <p log{t) be a Lamport cause of the 
violation determined by Definition [TJ] Let ad be a projected 
sublog of I, i.e., let ad ILp 1. We say that ad is the actual 
cause of violation t of pv if the following hold; 

1) (Sufficiency’) Pick any /. Let Cq = 
dummify(/, yf, E, Od,/) and let T be the 
set of traces starting from Cq whose logs 
contain ad as a projected sublog, i.e., T = 
{t' I is a trace starting from Cq and ad Ep log(^t')}. 
Then, T is non-empty and every trace in T has the 
violation pv, i.e, T C py. 

2) (Minimality’) No proper sublog of ad satisfies condi¬ 
tion [T] 

At the end of Phase 2, we obtain one or more sets of 
actions ad- These sets are deemed the independent actual 
causes of the violation t. 

The following theorem states that for all safety properties 
that are re-ordering closed, the Phase 1 and Phase 2 defini¬ 
tions always identify at least one Lamport and at least one 
actual cause. 

Theorem 1: Suppose py is reordering-closed and the 
complement of a safety property, i.e., RC(;/5y) and 
safety(-i;^y). Then, for every t G py: (1) Our Phase 
1 definition (Definition fT2l) finds a Lamport cause I, and 
(2) For every such Lamport cause I, the Phase 2 definition 
(Definition O finds an actual cause ad- 

Proof: (1) Pick any t G py- We follow the Phase 1 
definition. It suffices to prove that there is a log I <p log{t) 
that satisfies the sufficiency condition. Since safety(-i(/jy), 
there is a prefix to of t s.t. for all fi G U, to-ti G py - Choose 
I = log{to)- Since to is a prefix of t, I = logfio) <p log{t)- 
To prove sufficiency, pick any trace t' s.t. I <p log{t')- It 
suffices to prove t' G py- Since I <p logit'), for each 
i, log{t')\i = l\i ■ for some Let t" be the (unique) 
subsequence of t' containing all labels from the logs {('}. 
Consider the trace s = to-t"- First, s extends to, so s G py- 
Second, s ^ t' because logis)\i = l\i ■ = log{tQ)\i ■ 

logit")\i = logito ■ t")\i = logit')\i- Since RC((^y), t' G 
pv- 

(2) Pick any t G py and let I be a Lamport cause of 
t as determined by the Phase 1 definition. Following the 
Phase 2 definition, we only need to prove that there is at 
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least one ad I that satisfies the sufficiency’ condition. We 
choose ad = L To show sufficiency’, pick any /. Because 
ad = I, ad specifies an initial prefix of every A(i) and 
the transform dummify() has no effect on this prefix. First, 
we need to show that at least one trace t' starting from 
dummify(/, S, Od,/) satisfies ad Ep log{t'). For this, 
we can pick t' = t. Second, we need to prove that any trace 
t' starting from dummify(/, yf, S, Od,/) s.t. ad Ep log{t') 
satisfies t' G (pv- Pick such a E. Let to be the prefix of 
t corresponding to 1. Then, ^op(fo)|i = l\i for each i. It 
follows immediately that for each i, t'\i = to\i ■ t” for some 
t'l. Let t" be the unique subsequence of t' containing all 
labels from traces {<"}. Let s = to ■ t". First, because for 
each i, l\i = log{to)\i, I <p log{to) trivially. Because I is a 
Lamport cause, it satisfies the sufficiency condition of Phase 
1, so to G (fiv- Since safety(-'(/3y), and s extends to, s G <pv- 
Second, s ^ t' because log{s)\i = log{to)\i ■ log(t'')\i = 
log{f)\i and both s and t' are traces starting from the initial 
configuration dummify(/, yf, S, Od,/). Hence, by RC((/3y), 
t' G ipv- ■ 

Our Phase 2 definition identifies a set of program actions 
as causes of a violation. However, in some applications it 
may be necessary to ascribe thread identifiers (or programs) 
as causes. This can be straightforwardly handled by lifting 
the Phase 2 definition: A thread i (or A{i)) is a cause if one 
of its actions appears in ad- 

Definition 15 (Program Cause of Violation): Let Od be 
an actual cause of violation pv on trace t starting from 
(/, A, S). We say that the set X C / of thread identifiers is 
a cause of the violation if X = {i \ i appears in Od}. 

Remarks: We make a few technical observations about 
our definitions of cause. First, because Lamport causes 
(DefinitionfTSli are projected prefixes, they contain all actions 
that occur before any action that actually contributes to 
the violation. Many of actions in the Lamport cause may 
not contribute to the violation intuitively. Our actual cause 
definition filters out such “spurious” actions. As an example, 
suppose that a safety property requires that the value 1 never 
be sent on the network. The (only) trace of the program 
X = l\y = 2\z = 3; send(a:) violates this property. The 
Lamport cause of this violation contains all four actions of 
the program, but it is intuitively clear that the two actions 
y = 2 and z = 3 do not contribute to the violation. Indeed, 
the actual cause of the violation determined by Definition fT4l 
does not contain these two actions; it contains only x = 1 
and send(x), both of which obviously contribute to the 
violation. 

Second, our definition of dummification is based on a pro¬ 
gram transformation that needs line numbers. One possibly 
unwanted consequence is that our traces have line numbers 
and, hence, we could, in principle, specify safety properties 
that are sensitive to line numbers. However, our definitions 
of cause are closed under bijective renaming of line numbers, 
so if a safety property is insensitive to line numbers, the 


actual causes can be quotiented under bijective renamings 
of line numbers. 

Third, our definition of actual cause (Definition fT4l) 
separates actions whose return values are relevant to the 
violation from those whose return values are irrelevant for 
the violation. This is closely related to noninterference¬ 
like security definitions for information flow control, in 
particular, those that separate input presence from input 
content lf30l . Lamport causes (Definition [T2]l have a trivial 
connection to information flow: If an action does not occur 
in any Lamport cause of a violation, then there cannot be an 
information flow from that action to the occurrence of the 
violation. 

IV. Causes of Authentication Failures 

In this section, we model an instance of our running exam¬ 
ple based on passwords (Section [Till in order to demonstrate 
our actual cause definition. As explained in Section [III we 
consider a protocol session where Serverl, Userl, User2, 
User3 and multiple notaries interact over an adversarial net¬ 
work to establish access over a password-protected account. 
We describe a formal model of the protocol in our language, 
examine the attack scenario from Section [II] and provide a 
cause analysis using the definitions from Section mil 

A. Protocol Description 

We consider our example protocol with eight threads 
named {Serverl, Userl, Adversary, Notaryl, Notary2, 
Notary3, User2, User3}. In this section, we briefly describe 
the protocol and the programs specified by the protocol for 
each of these threads. For this purpose, we assume that we 
are provided a function Af : I Expressions such that Af{i) 
is the program that ideally should have been executing in the 
thread i. For each i, we call A/^(i) the norm for thread i. The 
violation is caused because some of the executing programs 
are different from the norms. These actual programs, called 
A as in Section mil are shown later. The norms are shown 
here to help the reader understand what the ideal protocol is 
and also to facilitate some of the development in Section [V] 
The appendix describes an expansion of this example with 
more than the eight threads considered here to illustrate our 
definitions better. The proof included in the appendix deals 
with timestamps and signatures. 

The norms in Figure [Hand the actuals in Figure [2] assume 
that Userl’s account (called acct in Serverl’s program) has 
already been created and that Userl’s password, pwd is 
associated with Userl’s user id, uid. This association (in 
hashed form) is stored in Serverl’s local state at pointer 
mem. The norm for Serverl is to wait for a request from 
an entity, respond with its (Serverl’s) public key, wait for 
a username-password pair encrypted with that public key 
and grant access to the requester if the password matches 
the previously stored value in Serverl’s memory at mem. 
To grant access, Serverl adds an entry into a private access 
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matrix, called P. (A separate server thread, not shown here, 
allows Userl to access its account if this entry exists in P.) 

The norm for Userl is to send an access request to 
Serverl, wait for the server’s public key, verify that key 
with three notaries and then send its password pwd to 
Serverl, encrypted under Serverl’s public key. On receiving 
Serverl’s public key, Userl initiates a protocol with the 
three notaries and accepts or rejects the key based on the 
response of a majority of the notaries. For simplicity, we 
omit a detailed description of this protocol between Userl 
and the notaries that authenticates the notaries and ensures 
freshness of their responses. These details are included in our 
appendix. In parallel, the norm for User2 is to generate and 
send a nonce to User3. The norm for User3 is to receive a 
message from User2, generate a nonce and send it to User2. 

Each notary has a private database of (publicjcey, prin¬ 
cipal) tuples. The notaries’ norms assume that this database 
has already been created correctly. When Userl sends a 
request with a public key, the notary responds with the 
principal’s identifier after retrieving the tuple corresponding 
to the key from its database. 

Notation: The programs in this example use several 
primitive functions Enc(fc,m) and Dec{k',m) denote 
encryption and decryption of message m with key k and 
k' respectively. Hash(m) generates the hash of term m. 
Sig(fc, m) denotes message m signed with the key k, paired 
with m in the clear. pub_key_i and pvt_key_i denote 
the public and private keys of thread i, respectively. For 
readability, we include the intended recipient i and expected 
sender j of a message as the first argument of send(i,m) 
and recv(j) expressions. As explained earlier, i and j are 
ignored during execution and a network adversary, if present, 
may capture or inject any messages. 

Security property: The security property of interest to 
us is that if at time u, a thread k is given access to account 
a, then k owns a. Specifically, in this example, we are 
interested in case a = acct and k = Userl. This can be 
formalized by the following logical formula, 

Vu, k. {acct, k) £ P{u) D {k = Userl) (1) 

Here, P{u) is the state of the access control matrix P for 
Serverl at time u. 

B. Attack 

As an illustration, we model the “Compromised Notaries” 
violation of SectionUT] The programs executed by all threads 
are given in Figure |2] Userl sends an access request to 
Serverl which is intercepted by Adversary who sends its 
own key to Userl (pretending to be Serverl). Userl checks 
with the three notaries who falsely verify Adversary’s public 
key to be Serverl’s key. Consequently, Userl sends the 
password to Adversary. Adversary then initiates a protocol 
with Serverl and gains access to the Userl’s account. Note 
that the actual programs of the three notaries attest that the 


Norm A/’(Serverl): 

1 : _ = recv(j); //access req from thread j 

2 : send(ji,pu&_fcej/_Serverl); //send public key to j 

3 : s = recv(/); //encrypted uid,pwd, thread id J 

4 : {uid,pwd, J) = T)ec{pvt_key_Ser\/erl, s); 

5 : t = hash{uid,pwd); 

assert (mem = t) //compare hash with stored value 

6 : insert(P, {acct, J)); 

Norm A/’(Userl): 

1 : send(Serverl); //access request 

2 : pub_key = recv(Serverl); //key from Serverl 

3 : send(Notaryl,pu6_fcej/); 

4 : send(Notary2,pu6_fcej/); 

5 : send(Notary3,pu6_fcej/); 

6 : Sig{pub_key, 11) = recv(Notaryl); //notaryl responds 

7 : Sig{pub_key, 12) = recv(Notary2); //notary2 responds 

8 : Sig{pub_key, 13) = recv(Notary3); //notary3 responds 
assert(At least two of {11,12,13} equal Serverl) 

9 : t — Enc{pub_key, {uid,pwd, Userl)); 

10 : send(Serverl,f); //send t to Serverl 

Norms A/'(Notaryl),A/'(Notary2),A/'(Notary3): 

// o denotes Notaryl, Notary2 or Notary3 

1 : pub_key = recv(/); 

2 : pr = KeyO'wrLer{pub_key); //lookup key owner 

3 : send{j,Sig{pvt_key_o, {pub_key,pr)); 

Norm A/’(User2): 

1 : send(User3); 

2 : _ = recv(User3); 

Norm A/’(User3): 

1 : _ = recv(User2); 

2 : send(User3); 

Figure 1. Norms for all threads. Adversary’s norm is the trivial empty 
program. 


public key given to them belongs to Serverl. In parallel, 
User2 sends a request to Serverl and receives a response 
from Serverl. Following this interaction, User2 interacts 
with User3, as in their norms. 

Figure [3 shows the expressions executed by each 
thread on the property-violating trace. For instance, 
the label ((Userl, 1), (Adversary, 1)) indicates that both 
Userl and Adversary executed the expressions with the 
line number 1 in their actual programs, which re¬ 
sulted in a synchronous communication between them, 
while the label (Adversary, 4) indicates the local ex¬ 
ecution of the expression at line 4 of Adversary’s 
program. The initial configuration has the programs; 
{^(Userl), ^(Serverl), ^(Adversary), yl(Notaryl), 
^(Notary2), ,4(Notary3), ^(User2), ,4.(User3)}. For this at¬ 
tack scenario, the concrete trace t we consider is such 
that log(f) is any arbitrary interleaving of the actions for 
X = {Adversary, Userl, User2, UserS, Serverl, Notaryl, 
Notary2, Notary3} shown in Figure Ha). Any such inter¬ 
leaved log is denoted log(f) in the sequel. At the end 
of this log, (accf, Adversary) occurs in the access control 







Actual ^(Serverl): 

1 : _ = recv(j); //access req from thread j 

2 : send(ji,p«&_fce 2 /_Serverl); //send public key to j 

3 : _ = recv(jf); //receive nonce from thread User2 

4 : send(/); //send signed nonce 

5 ; s = recv(j); //encrypted uid,pwd, thread id from j 

6 : {uid,pwd, J) — Vec{pvt_key_Serverl, s); 

7 : t = hash{uid,pwd); 

assert (mem = f)[A] //compare hash with stored value 

8 : insert(P, {acct, J)); 

Actual ^(Userl): 

1 : send(Serverl); //access request 

2 : pub_key = recv(Serverl); //key from Serverl 

3 : send(Notaryl,pM6_fcet/); 

4 : send(Notary2,pub_fcet/); 

5 : send(Notary3,pu&_fcet/); 

6 : Sig{pub_key, 11) = recv(Notaryl); //notaryl responds 

7 : Sig{pub_key, 12) = recv(Notary2); //notary2 responds 

8 : Sig{pub_key, 13) = recv(Notary3); //notaryS responds 
assert(At least two of {11,12,13} equal Serverl)[B] 

9 : f = Enc{pub_key, (uid,pwd, Userl)); 

10 : send(Serverl, f); //send t to Serverl 

Actual ^(Adversary) 

1 : recv(Userl); //mt&Yce.\A access req from Userl 

2 : send(Userl,pub_fcej/_A); //send key to User 

3 : s = recv(Userl); //pwd from Userl 

4 : (uid,pwd, Userl) = Dec{pvt_key_A, s); //decrypt pwd 

5 ; send(Serverl, uid); //access request to Serverl 

6 : pub_key = recv(Serverl); //Receive ServerUs public key 

7 : t = Eiic{pub_key, {uid,pwd, Adversary)); //encrypt pwd 

8 : send(Serverl,f); //pwd to Serverl 

Actuals A(Notaryl),^(Notary2),^^(NotaryS): 

// o denotes Notaryl, Notary2 or Notary3 

1 : pub_key = recv(j); 

2 ; send{j,Sig{pvt_key_o, (pu6_fcep, Serverl))); 

Actual ^(User2): 

1 ; send(Serverl); //send nonce to Serverl 

2 : _ = recv(Serverl); 

3 : send(User3); //forward nonce to User3 

4 : _ = recv(User3); 

Actual ^(User3): 

1 ; _ = recv(User2); 

2 : send(User2); //send nonce to User2 

Figure 2. Actuals for all threads. 


matrix P, but Adversary does not own acct. Hence, this log 
corresponds to a violation of our security property. 

Note that if any two of the three notaries had attested the 
Adversary’s key to belong to Serverl, the violation would 
still have happened. Consequently, we may expect three 
independent program causes in this example: {Adversary, 
Userl, Serverl, Notaryl, Notary2} with the action causes 
Qd as shown in Figure [He), (Adversary, Userl, Serverl, 
Notaryl, NotarySj with the actions and {Adversary, 
Userl, Serverl, Notary2, Notary3} with the actions a'^ 
where a'^ and a'^ can be obtained from aa (Figure EJ by 
considering actions for {Notaryl, Notary3} and {Notary2, 


Notary3} respectively, instead of actions for {Notaryl, 
Notary2}. Our treatment of independent causes follows the 
tradition in the causality literature. The following theorem 
states that our definitions determine exactly these three 
independent causes - one notary is dropped from each of 
these sets, but no notary is discharged from all the sets. This 
determination reflects the intuition that only two dishonest 
notaries are sufficient to cause the violation. Additionally, 
while it is true that all parties who follow the protocol 
should not be blamed for a violation, an honest party may 
be an actual cause of the violation (in both the common 
and the philosophical sense of the word), as demonstrated 
in this case study. This two-tiered view of accountability 
of an action by separately asserting cause and blame can 
also be found in prior work in law and philosophy Q, OTll . 
Determining actual cause is nontrivial and is the focus of 
this work. 

Theorem 2: Let / = {Userl, Serverl, Adversary, Notaryl, 
Notary2, NotaryS, User2, UserS} and E and A be as 
described above. Let f be a trace from (/, A, E) such that 
log{t)\i for each i G I matches the corresponding log 
projection from Figure EJa). Then, Definition [Tsl determines 
three possible values for the program cause X of violation 
t G ipv'- {Adversary, Userl, Serverl, Notaryl, Notary2}, 
{Adversary, Userl, Serverl, Notaryl, Notary3}, and 
{Adversary, Userl, Serverl, Notary2, Notary3} where the 
corresponding actual causes are ad, a'^ and a'^, respectively. 

It is instructive to understand the proof of this theorem, as 
it illustrates our definitions of causation. We verify that our 
Phase 1, Phase 2 definitions (Definitions [12] [141 [Ejl yield 
exactly the three values for X mentioned in the theorem. 

Lamport cause (Phase 1): We show that any I whose 
projections match those shown in Figure Etb) satisfies 
sufficiency and minimality. From Figure E{b), such an I 
has no actions for User3 and only those actions of User2 
that are involved in synchronization with Serverl. For all 
other threads, the log contains every action from t. The 
intuitive explanation for this I is straightforward: Since I 
must be a (projected) prefix of the trace, and the violation 
only happens because of insert in the last statement of 
Serverl’s program, every action of every program before 
that statement in Lamport’s happens-before relation must be 
in 1. This is exactly the I described in Figure Elb). 

Formally, following the statement of sufficiency, let T be 
the set of traces starting from Cq = (/, A, E) (Figure El 
whose logs contain I as a projected prefix. Pick any t' G 
T. We need to show t' G ipy. However, note that any f 
containing all actions in I must also add {acct, Adversary) 
to P, but Adversary Userl. Hence, t' G ipv- Further, I is 
minimal as described in the previous paragraph. 

Actual cause (Phase 2): Phase 2 (Definitions [T4| 
[13 determines three independent program causes for 
X: {Adversary, Userl, Serverl, Notaryl, Notary2}, 
{Adversary, Userl, Serverl, Notaryl, Notary3}, 
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(a)_ 

log(t) I Adversary 

{(Userl, 1), (Adversary, 1)), 
{(Adversary, 2), (Userl, 2)) 

((Userl, 10), (Adversary, 3)), 
(Adversary, 4), 

((Adversary, 5), (Serverl, 1)), 
((Serverl, 2), (Adversary, 6)), 
(Adversary, 7), 

((Adversary, 8), (Serverl, 5)), 
log(t) Iserverl 

((Adversary, 5), (Serverl, 1)), 
((Serverl, 2), (Adversary, 6)), 
((User2, 1), (Serverl, 3)), 
((Serverl, 4), (User2, 2)), 
((Adversary, 8), (Serverl, 5)), 
(Serverl, 6) 

(Serverl, 7) 

(Serverl, 8 ) 

log(*)|useri 

((Userl, 1), (Adversary, 1)), 
((Adversary, 2), (Userl, 2)) 
((Userl, 3), (Notaryl, 1)), 
((Userl,4), (Notary2, 1)), 
((Userl, 5), (Notary3, 1)), 
((Notaryl, 2), (Userl, 6)), 
((Notary2, 2), (Userl, 7)), 
((Notary3, 2), (Userl, 8)), 
(Userl, 9) 

((Userl, 10), (Adversary, 3)), 

log(t) I Notaryl 

((Userl, 3), (Notaryl, 1)), 
((Notaryl, 2), (Userl, 6)), 

log(t) I Notary2 

((Userl, 4), (Notary2, 1)), 
((Notary2, 2), (Userl, 7)), 

log(t) I NotaryS 

((Userl, 5), (Notary3, 1)), 
((Notary3, 2), (Userl, 8)), 

log(*)luser2 

((User2, 1), (Serverl, 3)), 
((Serverl, 4), (User2, 2)), 
((User2,3), (User3, 1)), 
((User2,4),(User3,2)), 


(b) 


((Userl, 1), (Adversary, 1)), 
((Adversary, 2), (Userl, 2)) 

((Userl, 10), (Adversary, 3)), 
(Adversary, 4), 

((Adversary, 5), (Serverl, 1)), 
((Serverl, 2), (Adversary, 6)), 
(Adversary, 7), 

((Adversary, 8), (Serverl, 5)), 

^ I Serverl 

((Adversary, 5), (Serverl, 1)), 
((Serverl, 2), (Adversary, 6)), 
((User2, 1), (Serverl, 3)), 
((Serverl, 4), (User2, 2)), 

((Adversary, 8), (Serverl, 5)), 
(Serverl, 6) 

(Serverl, 7) 

(Serv erl, 8) 

^ I Userl 

((Userl, 1), (Adversary, 1)), 
((Adversary, 2), (Userl, 2)) 
((Userl, 3), (Notaryl, 1)), 
((Userl,4), (Notary2, 1)), 
((Userl, 5), (Notary3, 1)), 
((Notaryl, 2), (Userl, 6)), 
((Notary2, 2), (Userl, 7)), 
((Notary3, 2), (Userl, 8)), 
(Userl, 9) 

((Userl, 10), (Adversary, 3)), 

^ iNotaryl 

((Userl, 3), (Notaryl, 1)), 
((Nota ryl, 2), (Userl, 6)), 

1 1 Notary2 

((Userl, 4), (Notary2, 1)), 
((Notary2, 2), (Userl, 7)), 

^ I NotaryS 

((Userl, 5), (Notary3, 1)), 
((Notary3, 2), (Userl, 8)), 

^ I User2 

(^er2, 1), (Serverl, 3)), 
((Serverl, 4), (User2, 2)), 


(C)_ 

lAdversary 

((Userl, 1), (Adversary, 1)), 
((Adversary, 2), (Userl, 2)) 
((Userl, 10), (Adversary, 3)), 
(Adversary, 4), 

((Adversary, 5), (Serverl, 1)), 
((Serverl, 2), (Adversary, 6)), 
(Adversary, 7), 

((Adversary, 8), (Serverl, 5)), 

I Serverl 

((Adversary, 5), (Serverl, 1)), 
((Serverl, 2), (Adversary, 6)), 


((Adversary, 8), (Serverl, 5)), 
(Serverl, 6) 

(Serverl, 7) 

(Serverl, 8) 

luserl 

((Userl, 1), (Adversary, 1)), 
((Adversary, 2), (Userl, 2)) 
((Userl, 3), (Notaryl, 1)), 
((Userl, 4), (Notary2, 1)), 

((Notaryl, 2), (Userl, 6)), 
((Notary2, 2), (Userl, 7)), 

(Userl, 9) 

((Userl, 10), (Adversary, 3)), 
^d I Notaryl 

((Userl, 3), (Notaryl, 1)), 
((Notaryl, 2), (Userl, 6)), 

Q-d I Notary2 

((Userl, 4), (Notary2, 1)), 
((Notary2, 2), (Userl, 7)), 


log(*)|User3 

((User2,3), (User3, 1)), 
((User2,4), (User3,2)), 


Figure 3. Left to Right: (a): log(t)|i for i £ 1. (b): Lamport cause I for Theorem[2| l\i = 0 for i G {UserS} as output by Definition 1121 (c): Actual 
cause 0(1 for Theorem[2] a^li = 0 for i £ {NotaryS, User2, UserS}. is a projected sublog of Lamport cause 1. 


and {Adversary, Userl, Serverl, Notary2, NotaryS} 
with the actual action causes given by ad-,a!^ and 
respectively in Figure [3] These are symmetric, 
so we only explain why ad satisfies Definition M 
(For this Ud, Definition [15] immediately forces 
X = {Adversary, Userl, Serverl, Notaryl, Notary2}.) 
We show that (a) ad satisfies sufficiency’, and (b) No 
proper sublog of ad satisfies sufficiency’ (minimality’). 
Note that ad is obtained from I by dropping NotaryS, User2 
and UserS, and all their interactions with other threads. 

We start with (a). Let ad be such that ad\i matches 
Figure Oc) for every i. Fix any dummifying func¬ 
tion /. We must show that any trace originating from 
dummify(/, E, Od,/), whose log contains ad as a pro¬ 
jected sublog, is in (pv- Additionally we must show 


that there is such a trace. There are two potential is¬ 
sues in mimicking the execution in ad starting from 
dummify(/,,/!, E, Od, /) — first, with the interaction be¬ 
tween Userl and NotaryS and, second, with the interaction 
between Serverl and User2. For the first interaction, on 
line 5, ^(Userl) (Figure |2]i synchronizes with NotaryS 
according to I, but the synchronization label does not exist 
in Qd- However, in dummify(/, yl, E, a^,/), the recv() 
on line 8 in ^(Userl) is replaced with a dummy value, 
so the execution from dummify(/, ,4,, E, a^,/) progresses. 
Subsequently, the majority check (assertion [B]) succeeds as 
in /, because two of the three notaries (Notaryl and Notary2) 
still attest the Adversary’s key. A similar observation can be 
made about the interaction between Serverl and User2. 

Next we prove that every trace starting from 
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dummify(/, S, Ud,/), whose log contains Ud (Figure 
as a projected sublog, is in tpv- Fix a trace t' with log I'. 
Assume /' contains ad- We show t' G (py as follows; 

1) Since the synchronization labels in I' are a superset of 
those in ad, Server 1 must execute line 8 of its program 
,4(Serverl) in t'. After this line, the access control 
matrix P contains {acct, J) for some J. 

2) When ^(Serverl) writes (x, J) to P at line 8, then J is 
the third component of a tuple obtained by decrypting 
a message received on line 5. 

3) Since the synchronization projections on I' are a su¬ 
perset of ad, and on ad (Serverl, 5) synchronizes with 
(Adversary, 8), J must be the third component of an 
encrypted message sent on line 8 of ,4(Adversary). 

4) The third component of the message sent on line 8 by 
Adversary is exactly the term “Adversary”. (This is easy 
to see, as the term “Adversary” is hardcoded on line 7.) 
Hence, J = Adversary. 

5) This immediately implies that t' G ipy since 
{acct, Adversary) G P, but Adversary ^ Userl. 

Last, we prove (b) — that no proper subsequence of ad 
satisfies sufficiency’. Note that ad (Figure [3c)) contains 
exactly those actions from I (Figure O on whose returned 
values the last statement of Serverl’s program (Figure [2]) is 
data or control dependent. Consequently, all of ad as shown 
is necessary to obtain the violation. 

(The astute reader may note that in Figure [2] there is no 
dependency between line 1 of Serverl’s program and the 
insert statement in Serverl. Hence, line 1 should not be 
in ad- While this is accurate, the program in Figure |3 is a 
slight simplification of the real protocol, which is shown in 
the appendix. In the real protocol, line 1 returns a received 
nonce, whose value does influence whether or not execution 
proceeds to the insert statement.) 

V. Towards Accountability 

In this section, we discuss the use of our causal analysis 
techniques for providing explanations and assigning blame. 

A- Using Causality for Explanations 

Generating explanations involves enhancing the epistemic 
state of an agent by providing information about the cause 
of an outcome ll^ . Automating this process is useful for 
several tasks such as planning in Al-related applications and 
has also been of interest in the philosophy community li^ . 

Causation has also been applied for explaining counter 
examples and providing explanations for errors in model 
checking ||33, Ea, IMl, 1321 where the abstract nature of 
the explanation provides insight about the model. 

In prior work, Halpern and Pearl have defined explanation 
in terms of causality ll^ . A fact, say E, constitutes an 
explanation for a previously established fact F in a given 
context, if had E been true then it would have been a 
sufficient cause of the established fact F . Moreover, having 


this information advances the prior epistemic state of the 
agent seeking the explanation, i.e. there exists a world (or a 
setting of the variables in Halpern and Pearl’s model) where 
F is not true but E is. 

Our definition of cause (Section inB could be used to 
explain violations arising from execution of programs in a 
given initial configuration. Given a log I, an initial configu¬ 
ration Co, and a violation py, our definition would pinpoint 
a sequence of program actions, ad, as an actual cause of the 
violation on the log. ad would also be an explanation for 
the violation on I if having this causal information advances 
the epistemic knowledge of the agent. Note that there could 
be traces arising from the initial configuration where the 
behavior is inconsistent with the log. Knowing that ad is 
consistent with the behavior on the log and that it is a cause 
of the violation would advance the agent’s knowledge and 
provide an explanation for the violation. 

B. Using Causality for Blame Attribution 

Actual causation is an integral part of the prominent the¬ 
ories of blame in social psychology and legal settings lIMll . 
EH, ED, Eol. Most of these theories provide a com¬ 
prehensive framework for blame which integrates causality, 
intentionality and foreseeability EH, ED, ED- These the¬ 
ories recognize blame and cause as interrelated yet distinct 
concepts. Prior to attributing blame to an actor, a causal 
relation must be established between the actor’s actions and 
the outcome. However, not all actions which are determined 
as a cause are blameworthy and an agent can be blamed 
for an outcome even if their actions were not a direct cause 
(for instance if an agent was responsible for another agent’s 
actions). In our work we focus on the first aspect where we 
develop a theory for actual causation and provide a building 
block to find blameworthy programs from this set. 

We can use the causal set output by the definitions in 
Section |III| and further narrow down the set to find blame¬ 
worthy programs. Note that in order to use our definition as a 
building block for blame assignment, we require information 
about a) which of the executed programs deviate from the 
protocol, and b) which of these deviations are harmless. 
Some harmless deviants might be output as part of the 
causal set because their interaction is critical for the violation 
to occur. Definition [T3 below provides one approach to 
removing such non-blameworthy programs from the causal 
set. In addition we can filter the norms from the causal set. 

For this purpose, we use the notion of protocol specified 
norms J\f introduced in Section|IV] We impose an additional 
constraint on the norms, i.e., in the extreme counterfactual 
world where we execute norms only, there should be no 
possibility of violation. We call this condition necessity- 
Conceptually, necessity says that the reference standard 
(norms) we employ to assign blame is reasonable. 

Definition 16 (Necessity condition for norms): Given 
{I,'E,Af,(py), we say that Nf satisfies the necessity 
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condition w.r.t. tpv if for any trace t' starting from the 
initial configuration {I,N, S), it is the case that t' ^ (fv- 

We can use the norms M and the program cause X with its 
corresponding actual cause ad from Phase 2 (Definitions fT4l 
Ell, in order to determine whether a program is a harmless 
deviant as follows. Definition [IT] presents a sound (but not 
complete) approach for identifying harmless deviants. 

Definition 17 (Harmless deviant): Let X he a program 
cause of violation V and ad be the corresponding actual 
cause as determined by Definitions [14] and [Ts] We say that 
the program corresponding to index i G X is a harmless 
deviant w.r.t. trace t and violation (pv if A{i) is deviant 
(i.e. A{i) 7 ^ A/’(i)) and ad\i is a prefix of 

For instance in our case study (Section IIVI) . Theorem |2] 
outputs X and ad (Figure|3|l as a cause. X includes Serverl. 
Considering Serverl’s norm (Figure [T]i, .A{Server} will be 
considered a deviant, but according to Definition [T7| Serverl 
will be classified as a harmless deviant because Odlserveri 
is a prefix of Af(Serverl). Note that in order to capture 
blame attribution accurately, we will need a richer model 
which incorporates intentionality, epistemic knowledge and 
foreseeability, beyond causality. 

VI. Related Work 

Currently, there are multiple proposals for providing ac¬ 
countability in decentralized multi-agent systems 
ifTTl . Q, Ho), im, (Ha, ga, gl. Although the intrinsic 
relationship between causation and accountability is often 
acknowledged, the foundational studies of accountability do 
not explicitly incorporate the notion of cause in their formal 
definition or treat it as a blackbox concept without explicitly 
defining it. Our thesis is that accountability is not a trace 
property since evidence from the log alone does not provide 
a justifiable basis to determine accountable parties. Actual 
causation is not a trace property; inferring actions which 
are actual causes of a violating trace requires analyzing 
counterfactual traces (see our sufficiency conditions). Ac¬ 
countability depends on actual causation and is, therefore, 
also not a trace property. 

On the other hand, prior work on actual causation in 
analytical philosophy and AI has considered counterfactual 
based causation in detail |[T3l . 1(141 . |[T9l . Il20l . lITSl . 1(151 . 
These ideas have been applied for fault diagnosis where 
system components are analyzed, but these frameworks do 
not adequately capture all the elements crucial to model 
a security setting. Executions in security settings involve 
interactions among concurrently running programs in the 
presence of adversaries, and little can be assumed about the 
scheduling of events. We discuss below those lines of work 
which are most closely related to ours. 

Accountability: Kiisters et al HD define a protocol P 
with associated accountability constraints that are rules of 
the form: if a particular property holds over runs of the proto¬ 
col instances then particular agents may be blamed. Further, 


they define a judge J who gives a verdict over a run r of an 
instance tt of a protocol P, where the verdict blames agents. 
In their work, Kiisters et al assume that the accountability 
constraints for each protocol are given and complete. They 
state that the judge J should be designed so that J’s verdict 
is fair and complete w.r.t. these accountability constraints. 
They design a judge separately for every protocol with a 
specific accountability property. Kiisters et al.’s definition of 
accountability has been successfully applied to substantial 
protocols such as voting, auctions, and contract signing. 
Our work complements this line of work in that we aim to 
provide a semantic basis for arriving at such accountability 
constraints, thereby providing a justification for the blame 
assignment suggested by those constraints. Our actual cause 
definition can be viewed as a generic judging procedure that 
is defined independent of the violation and the protocol. 
We believe that using our cause definition as the basis for 
accountability constraints would also ensure the minimality 
of verdicts given by the judges. 

Backes et al Cl define accountability as the ability to 
show evidence when an agent deviates. The authors analyze 
a contract signing protocol using protocol composition logic. 
In particular, the authors consider the case when the trusted 
third-party acts dishonestly and prove that the party can be 
held accountable by looking at a violating trace. This work 
can be viewed as a special case of the subsequent work of 
Kiisters et al. CD where the property associated with the 
violating trace is an example of an accountability constraint. 

Feigenbaum et al ll23l . Il24l also propose a definition 
of accountability that focuses on linking a violation to 
punishment. They use Halpern and Pearl’s definition 1(131 . 
Cl of causality in order to define mediated punishment, 
where punishment is justified by the existence of a causal 
chain of events in addition to satisfaction of some utility 
conditions. The underlying ideas of our cause definition 
could be adapted to their framework to instantiate the 
causality notion that is currently used as a black box in 
their definition of mediated punishment. One key difference 
is that we focus on finding program actions that lead to the 
violation, which could explain why the violation happened 
while they focus on establishing a causal chain between 
violation and punishment events. 

Causation for blame assignment: The work by Barth 
et al II 42 I provides a definition of accountability that uses the 
much coarser notion of Lamport causality, which is related 
to Phase 1 of our definition. However, we use minimality 
checks and filter out progress enablers in Phase 2 to obtain 
a finer determination of actual cause. 

Gossler et al’s work l(43l . 141 considers blame assignment 
for safety property violations where the violation of the 
global safety property implies that some components have 
violated their local specifications. They use a counterfactual 
notion of causality similar in spirit to ours to identify a 
subset of these faulty components as causes of the violation. 
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The most recent work in this line applies the framework to 
real-time systems specified using timed automata ll46ll . 

A key technical difference between this line of work 
and ours is the way in which the contingencies to be 
considered in counterfactual reasoning are constructed. We 
have a program-based approach to leverage reasoning meth¬ 
ods based on invariants and program logics. Gossler et al 
assume that a dependency relation that captures information 
flow between component actions are given and construct 
their contingencies using the traces of faulty components 
observed on the log as a basis. A set of faulty components 
is the necessary cause of the violation if the violation would 
disappear once the traces of these faulty components are 
modified to match the components’ local specifications. 
They determine the longest prefixes of faulty components 
that satisfy the specification and replace the faulty suffixes 
with a correct one. Doing such a replacement without taking 
into account its impact on the behavior of other components 
that interact with the faulty components would not be satis¬ 
factory. Indeed, Wang et al M describe a counterexample 
to Gossler et al’s work ll43l where all causes are not found 
because of not being able to completely capture the effect 
of one component’s behavior on another’s. The most recent 
definitions of Gossler et al ll45l . ll46l address this issue by 
over approximating the parts of the log affected by the faulty 
components and replacing them with behavior that would 
have arisen had the faulty ones behaved correctly. 

In constructing the contingencies to consider in counter- 
factual reasoning, we do not work with individual traces as 
Gossler et al. Instead, we work at the level of programs 
where “correcting” behavior is done by replacing program 
actions with those that do not have any effect on the violation 
other than enabling the programs to progress. The relevant 
contingencies follow directly from the execution of programs 
where such replacements have been done, without any need 
to develop additional machinery for reconstructing traces. 
Note also that we have a sufficiently fine-grained definition 
to pinpoint the minimal set of actions that make the compo¬ 
nent a part of the cause, where these actions may a part be 
of faulty or non-faulty programs. Moreover, we purposely 
separate cause determination and blame assignment because 
we believe that in the security setting, blame assignment is 
a problem that requires additional criteria to be considered 
such as the ability to make a choice, and intention. The 
work presented in this paper focuses on identifying cause as 
a building block for blame assignment. 

VII. Conclusion 

We have presented a first attempt at defining what it 
means for a sequence of program actions to be an actual 
cause of a violation of a security property. This question is 
motivated by security applications where agents can exercise 
their choice to either execute a prescribed program or deviate 
from it. While we demonstrate the value of this definition by 


analyzing a set of authentication failures, it would be inter¬ 
esting to explore applications to other protocols in which ac¬ 
countability concerns are central, in particular, protocols for 
electronic voting and secure multiparty computation in the 
semi-honest model. Another challenge in security settings is 
that deviant programs executed by malicious agents may not 
be available for analysis; rather there will be evidence about 
certain actions committed by such agents. A generalized 
treatment accounting for such partial observability would be 
technically interesting and useful for other practical appli¬ 
cations. This work demonstrates the importance of program 
actions as causes as a useful building block for several 
such applications, in particular for providing explanations, 
assigning blame and providing accountability guarantees for 
security protocols. 
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Appendix 


A. Operational Semantics 

Selected rules of the operational semantics of the programming language L are shown below. 


T^T‘ 


eval t t' 


eval t' t" 


red-end 


red-act 


I ,t ,a ^ I ,t' ,a' 



eval t true 


red-assert 


7,(assert(f);e) ,cr A I ,e,(T 


Internal reduction 



red-config 


Communication action 


eval t t 


■ {{bs ■■ X = send(f));es) ,o-s>, (/r , ((br : y = recv()); e^), ffr), ■ • • 


red-comm 



B. Case study: Compromised notaries attack 

We model an instance of our running example based on passwords in order to demonstrate our actual cause definition. As 
explained in Section [III we consider a protocol session where Serverl, Userl, User2, User3 and multiple notaries interact 
over an adversarial network to establish access over a password-protected account. In parallel for this scenario, we assume 
the log also contains interactions of a second server (Server2), one notary (Notary4, not contacted by Userl, User2 or User3) 
and another user (User4) who follow their norms for account access. These threads do not interact with threads {Userl, 
Serverl, Notaryl, Notary2, Notary3, Adversary, User2, User3}. The protocol has been described in detail below. 

1) Protocol Description: We consider our example protocol with eleven threads named (Serverl, Userl, User2, User3, 
Adversary, Notaryl, Notary2, Notary3, Notary4, Server2, User4}. The norms for all these threads, except Adversary are 
shown in Figure |4] The actual violation is caused because some of the executing programs are different from the norms. 
These actual programs, called A as in Section |III] are shown later. The norms are shown here to help the reader understand 
what the ideal protocol is. 

In this case study, we have two servers (Serverl, Server2) running the protocol with two different users (Userl, User4) and 
each server allocates account access separately. The norms in Figured assume that Userl’s and User4’s accounts (called accti 
and acct 2 in Serverl’s and Server2’s norm respectively) have been created already. Userl’s password, pwdi is associated 
with Userl’s user id uidl. Similarly User4’s password pwd 2 is associated with its user id uid2. This association (in hashed 
form) is stored in Serverl’s local state at pointer memi (and at mem 2 for Server2). The norm for Serverl is to wait for 
a request from an entity, respond with its public key, then wait for a password encrypted with that public key and grant 
access to the requester if the password matches the previously stored value in Serverl’s memory at memi. To grant access, 
Serverl adds an entry into a private access matrix, called Pi. (A separate server thread, not shown here, allows Userl to 
access its resource if this entry exists in Pi.) 

The norm for Userl is to send an access request to Serverl, wait for the server’s public key, verify that key with three 
notaries and then send its password pwdi to Serverl, encrypted under Serverl’s public key. On receiving Serverl’s public 
key, Userl initiates a protocol with the three notaries and accepts or rejects the key based on the response of a majority of 
the notaries. 

The norm for User4 is the same as that for Userl except that it interacts with Server2. Note that User4 only verifies the 
public key with one notary, Notary4. The norm for Server2 is the same as that for Serverl except that it interacts with 
User4. 

In parallel, the norm for User2 is to generate and send a nonce to User3. The norm for User3 is to receive a message 
from User2, generate a nonce and send it to User2. 
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Each notary has a private database of (publicjcey, principal) tuples. The norms here assume that this database has already 
been created correctly. When Userl or User4 send a request with a public key, the notary responds with the principal’s 
identifier after retrieving the tuple corresponding to the key in its database. (Note that, in this simple example, we identify 
threads with principals, so the notaries just store an association between public keys and their threads.) 

2) Preliminaries: 

Notation: The programs in this example use several primitive functions C,. Enc(fc, m) and Dec(/c', m) denote encryption 
and decryption of message m with key k and k' respectively. Hash(m) generates the hash of term m. Sig(fc, m) denotes 
message m signed with the key k, paired with m in the clear. pub_key_i and pvt_key_i denote the public and private keys 
of thread i, respectively. For readability, we include the intended recipient i and expected sender j of a message as the first 
argument of send(i, m) and recv(j) expressions. As explained earlier, i and j are ignored during execution and a network 
adversary, if present, may capture or inject any messages. Send(i, j,m) @ u holds if thread i sends message m to thread j 
at time u and Recv(i, j, m) @ u hold if thread i receives message m from thread j at time u. Pi{u) and P 2 {u) denotes the 
tuples in the permission matrices at time u. Initially Pi and P 2 do not contain any access permissions. 

Assumptions: (Al) 

HonestThread(Serverl, ,4(Serverl)) 

We are interested in security guarantees about users who create accounts by interacting with the server and who do not 
share the generated password or user-id with any other principal except for sending it according to the roles specified in the 
program given below. 

(A2) 

HonestThread(Userl, ,4(Userl)) 

(A3) 

HonestThread(Adversary, ,4( Adversary)) 

(A4) 

HonestThread(Notaryl, ,4(Notaryl)) 

(A5) 

HonestThread(Notary2, ,A(Notary2)) 

(A6) 

HonestThread(Notary3, ,4(Notary3)) 

(A7) 

HonestThread(Notary4, ,4(Notary4)) 

(A8) 

HonestThread(Server2, ,4(Server2)) 

(A9) 

HonestThread(User4, ,4(User4)) 

(AlO) 

HonestThread(User2, ,4(User2)) 

(All) 

HonestThread(User3, ,4(User3)) 

A principal following the protocol never shares its keys with any other entity. We also assume that the encryption scheme 
in semantically secure and non-malleable. Since we identify threads with principals therefore each of the threads are owned 
by principals with the same identifier, for instance Serverl owns the thread that executes the program A(Serverl). 

(Startl) 

Start{i) @ —00 

where i refers to all the threads in the set described above. 
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Norm A/’(Serverl): 

1 : (uidl,nl) = recv(j); //access req from thread j 

2 : n2 = new; 

3 : send(/, (puti_fcey_Serverl, n2, nl)); //sign and send public key 

4 : si = recv(/); //encrypted uidl,pwdl from j, alongwith its thread id J 

5 : {n3,uidl,pwdl, J) = Dec{pvt_key_Serverl, si); 

6 : t — Hash(uidl,p«jdl); 

assert(memi = t) //compare hash with stored hash value for same uid 

7 : insert(Pi, (accfi, J)); 

Norm A/’(Userl): 

1 : nl = new; 

2 ; send(Serverl, (uidl,nl)); //access request 

3 : {pub_keyl,n2,nl) = recv(j); //key from j 

4 : n3, n4, n5 = new; 

5 : send(Notaryl,pub_fcet/l, n3); 

6 : send(Notary2,pub_fcet/l, n4); 

7 : send(Notary3, pu&_fcet/l, n5); 

8 ; Sig(pt;f_fcep_Notaryl, (pw&_fcepl, il, n3)) = recv(Notaryl); //notaryl responds 

9 ; Sig(piif_fcep_Notary2, {puh_keyl,l2,n‘l)) = recv(Notary2); //notary2 responds 

10 : Sig(pt;f_fcei/_Notary3, (pu&_fcei/l, i3, n5)) = recv(Notary3); //notary3 responds 
assert(At least two of {11,12,13} equal Serverl) 

11 : f = Enc(pitt(_fce2/1, n2, (uidl,pwdl, Userl)); 

12 : send(Serverl, f); //send t to Serverl; 


Norms A/’(Notaryl), A/’(Notary2), A/’(Notary3), A/’(Notary4): 

// o denotes Notaryl, Notary2, Notary3 or Notary4 

1 : {pub_key,nl) = recv(j); 

2 : pr — Key0wner(pu6_fcep); //lookup key owner 

3 ; send{j,Sig{pvt_key_o,{pub_key,pr,nl))); //signed certificate; 

Norm A/’(Server2): 

1 ; {uid2,nl) = recv(/); //access req from thread j 

2 : n2 = new; 

3 : send(/, (pu6_feep_Server2, n2, nl)); 

4 : si = recv(j); //encrypted uid2,pwd2 from j, alongwith its thread id J 

5 : {n2,uid2,pwd2, J) — Dec{pvt_key_Server2, si); 

6 : t = Ea.sh.{uid2,pwd2); 

assert(mem 2 — t) //compare hash with stored hash value for same uid 

7 : insert(P 2 , {acct 2 , J)); 

Norm jV'(User4): 

1 : nl = new; 

2 : send(Server2, (uid2,nl)); //access request 

3 : pub_key,n2, nl = recv(y); //key from j 

4 : n3 = new; 

5 : send(Notary4,pu6_fcet/, n3); 

6 : Sig(pt)f_fcep_Notary4, {pub_key,ll,n3)) = recv(Notary4); //notary4 responds 
assert({ll} equals Server2) 

7 : t = Enc{pub_key,n2, {md2,pwd2, User4)); 

8 : send(Server2,f); //send t to Server2; 

Norm A/’(User2): 

1 : nl = new; 

2 ; send(User3, (nl)); 

3 : Slg{pvt_key_j, (n2,nl)) = recv(User3); 4 : 

Norm A/’(User3): 

1 : nl = recv(User2); 

2 : n2 = new; 

3 : send(User3, Sig(pr;f_fcet/_User3, (n2,nl))); 

Figure 4. Norms for Serverl, Userl, Server2, User4, User2, User3 and the notaries. Adversary’s norm is the trivial empty program. 
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Security property: The security property of interest to us is that if at time u, a thread k is given access to account a, 
then k owns a. Specifically, in this example, we are interested in the a = accti and k = Userl. This can be formalized by 
the following logical formula, ^pv'- 


'^u,k. {accti, k) G ^i(u) D (k = Userl) (2) 

Here, Pi(u) is the state of the access control matrix Pi for Serverl at time u. 

The actuals for all threads are shown in Figure |5] and |6] 

3) Attack: As an illustration, we model the “Compromised Notaries” violation of Section |II] The programs executed by 
all threads are given in Figures |5] and |6] Userl sends an access request to Serverl which is intercepted by Adversary who 
sends its own key to Userl (pretending to be Serverl). Userl checks with the three notaries who falsely verify Adversary’s 
public key to be Serverl’s key. Consequently, Userl sends the password to Adversary. Adversary then initiates a protocol 
with Serverl and gains access to the Userl’s account. Note that the actual programs of the three notaries attest that the 
public key given to them belongs to Serverl. In parallel, User2 sends a request to Serverl and receives a response from 
Serverl. Following this interaction, User2 interacts with User3, as in their norms. User4, Server2 and Notary4 execute their 
actuals in order to access the account acct 2 as well. 

Figure [T] shows the expressions executed by each thread on the property-violating trace. For instance, the label 
((Userl, 1), (Adversary, 1)) indicates that both Userl and Adversary executed the expressions with the line number 1 
in their actual programs, which resulted in a synchronous communication between them, while the label (Adversary, 4) 
indicates the local execution of the expression at line 4 of Adversary’s program. The initial configuration has the programs: 
{,4(Userl), ,4,(Serverl), .A(Adversary), ,4,(Notaryl), ^(Notary2), .A(Notary3), ,4(User2), ,4,(User3), ^(User4), ^(Server2), 
^(Notaryd)}. For this attack scenario, the concrete trace t we consider is such that log(f) is any arbitrary in¬ 
terleaving of the actions for Xi = {Adversary, Userl, Serverl, Notaryl, Notary2, Notary3, User2, User3} and X 2 = 
{Server2, User4, Notaryd} shown in Figure |3 a) and Figured Any such interleaved log is denoted log(f) in the sequel. At 
the end of this log, (accf 1 , Adversary) occurs in the access control matrix Pi, but Adversary does not own accti. Hence, 
this log corresponds to a violation of our security property. 

Note that, if any two of the three notaries had attested the Adversary’s key to belong to Serverl, the violation would have 
still happened. Consequently, we may expect three independent program causes in this example: {Adversary, Userl, Serverl, 
Notaryl, Notary2} with the action causes ad as shown in Figure|3c), {Adversary, Userl, Serverl, Notaryl, NotaryS} with 
the actions a(j, and {Adversary, Userl, Serverl, Notary2, NotaryS} with the actions a'^ where and a'^ can be obtained 
from ad (Figure|7{c)) by considering actions for {Notaryl, Notary3} and {Notary2, Notary3} respectively, instead of actions 
for {Notaryl, Notary2}. The following theorem states that our definitions determine exactly these three independent causes. 

Theorem 3: Let I = {Userl, Serverl, Adversary, Notaryl, Notary2, Notary3, Notaryd, Server2, User4, User2, User3}, and 
E and A be as described above. Let f be a trace from (/, A, E) such that log{t)\i for each i € I matches the corresponding 
log projection from Figures |2ta) and |8] Then, Definition [15] determines three possible values for the program cause X of 
violation t G (pv- {Adversary, Userl, Serverl, Notaryl, Notary2}, {Adversary, Userl, Serverl, Notaryl, Notary3}, and 
{Adversary, Userl, Serverl, Notary2, Notary3} where the corresponding actual causes are ad,a'^ and a'^ respectively. 

It is instructive to understand the proof of this theorem, as it illustrates our definitions of causation. We verify that our 
Phase 1 and Phase 2 definitions (Definitions [T2| [T4| [TSl l yield exactly the three values for X mentioned in the theorem. 

Lamport cause (Phase 1): We show that any / whose projections match those shown in Figure |7{b) satisfies sufficiency 
and minimality. From Figure |71b), such an I has no actions for User3, User4, Notary4, Server2 and only those actions of 
User2 that are involved in synchronization with Serverl. For all other threads, the log contains every action from t. The 
intuitive explanation for this I is straightforward: Since I must be a (projected) prefix of the trace, and the violation only 
happens because of insert in the last statement of Serverl’s program, every action of every program before that statement 
in Lamport’s happens-before relation must be in 1. This is exactly the I described in Figure |7{b). 

Formally, following the statement of sufficiency, let T be the set of traces starting from Co = {I, A, E) (Figure jSjl whose 
logs contain I as a projected prefix. Pick any t' € T. We need to show t' G pv- However, note that any t' containing 
all actions in I must also add (accfi, Adversary) to Pi, but Adversary Userl. Hence, t' G pv- Further, I is minimal as 
described in the previous paragraph. 

Actual cause (Phase 2): Phase 2 (Definitions [T4| [TSl l determines three independent program causes for X: 
{Adversary, Userl, Serverl, Notaryl, Notary2}, {Adversary, Userl, Serverl, Notaryl, Notary3}, and {Adversary, Userl, 
Serverl, Notary2, Notary3} with the actual action causes given by ad,a'j^ and a'^, respectively in Figure |7jc). These 
are symmetric, so we only explain why ad satisfies Definition [T4| (For this ad. Definition [15] immediately forces 
X = {Adversary, Userl, Serverl, Notaryl, Notary2}.) We show that (a) ad satisfies sufficiency’, and (b) No proper sublog 
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Actual ^(Adversary) 

1 : {uidl,nl) = recv(y); //intercept req from Userl 

2 : n2 = new; 

3 : send(Userl, (pub_fcet/_Adversaryl, n2, nl)); //send key to Userl 

4 : s = recv(Userl); //pwd from User 

5 : n2, uidl,p'wdl, Userl = Dec(pr;f_fcet/_Adversary, s); //decrypt pwd; 

6 : n3 = new; 

7 : send(Serverl, (uidl,n3)); //access request to Server 

8 : pub_key,n4,n3 = recv(Serverl); 

9 :t = Enc{pub_key, (n4, uidl, ptndl, Adversary)); //encrypt pwd 

10 : send(Serverl, f); //pwd to Serverl 


Actuals ^(Notaryl), ^(Notary2), ^(Notary3): 

// o denotes Notary 1, Notary2 or Notary3 

1 : (pii6_fcei/_Adversary,nl) = recv(ji); 

2 : send(j, Sig{pvt_key_o, (p«fo_fcej/_Adversary, Serverl, nl)); //signed certificate to /; 


Actual ^(Serverl): 

1 : (Midl,nl) = recv(/); //access req from thread j 

2 : n2 = new; 

3 : send(/, (pu6_fcep_Serverl, n2, nl)); 

4 : n4 = recv(y); //receive nonce from thread User2 

5 : n5 = new; 

6 ; send(j, Sig(pt;f_fcep_Serverl, (n5,n4))); 

7 : si = recv(j); //encrypted uidl,pwdl from j, alongwith its thread id J 

8 : {nSjUidljpwdl, J) — T)ec{pvt_key_Seryerl, si); 

9 : t = Ea.sh.{uidl,pwdl)-, 

assert(memi = f)[A] //compare hash with stored hash value for same uid 

10 : insert(Pi, {accti, J)); 


Actual ^(Userl): 

1 : nl = new; 

2 ; send(Serverl, (uidl,nl)); //access request 

3 : {pub_key,n2,nl) = recv(j); //key from j 

4 : n3, n4, n5 = new; 

5 : send(Notaryl,pub_fcet/, n3); 

6 : send(Notary2,pub_fcet/, n4); 

7 : send(Notary3,pM&_fcet/, n5); 

8 : Sig(pi;f_fcep_Notaryl, (pMb_fcep, Zl, n3)) = recv(Notaryl); //notaryl responds 

9 ; Sig(piit_fcep_Notary2, {pub_key,l2,n4)) = recv(Notary2); //notary2 responds 

10 : Sig(pwf_fcet/_Notary3, (pu&_fcep, Z3, n5)) = recv(Notary3); //notaryS responds 
assert(At least two of{Zl, 12, Z3}equal Serverl); [B] / / 

11 : f = Eiic(pub_key,n2, {uidl,pwdl, Userl)); 

12 : send(Serverl, f); //send t to Serverl; 


Actual ^(User2): 

1 : nl = new; 

2 : send(Serverl, (nl)); 

3 : Sig(pt;f_fcep, (n2, nl)) = recv(Serverl); 

4 ; send(User3, (n2)); 

5 ; Sig{pub_key,n3,n2) = recv(User3); 


Actual ^(User3): 

1 : nl = recv(User2); 

2 : n2 = new; 

3 : send(User3, Sig(pr;f_fcep_User3, n2, nl)); 

Figure 5. Actuals for Adversary, Notaryl, Notary2, Notary!, Serverl, Userl, User2, User! 
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Actual ^(Server2): 

1 : (uid2,nl) = recv(j); //access req from thread j 

2 : n2 = new; 

3 : send{j,{pub_key_Ser\/er2,n2,nl)); 

4 : si = recv(jf); //encrypted uid2,pwd2 from j, alongwith its thread id J 

5 ; {n2,uid2,pwd2, J) — T)ec{pvt_key_Seryer2, si); 

6 : t — Hash(uid2,pmd2); 

assert(mem 2 = t) //(C)compare hash with stored hash value for same uid 

7 : insert(P 2 , (accf 2 , J)); 


Actual ^(User4): 

1 : nl = new; 

2 : send(Server2, (utd2,nl)); //access request 

3 : Sig(j}ub_key,n2,nl) = recv(j); //key from j 

4 : n3 = new; 

5 : send(Notary4,pM&_fcet/, n3); 

6 : Sig(pnf_fce2/_Notary4, {pub_key,ll,n3)) = recv(Notary4); //notary4 responds 
assert({ll} equals Server2)(Z)) 

7 : t = Enc{pub_key,n2, {uid2,pwd2, User4)); 

8 : send(Server2,f); //send t to Server2; 


Actual ^(Notary4): 

// o denotes Notary 1, Notary2, Notary3 or Notary4 

1 : {pub_key,nl) = recv(j); 

2 : pr — Key0wner(pu6_fcei/); //lookup key owner 

3 ; send{j,Sig{pvt_key_o,{pub_key,pr,nl))); //signed certificate; 

Figure 6. Actuals for Server2, User4, Notary4 


of ad satisfies sufficiency’ (minimality’). Note that ad is obtained from I by dropping Notary3, User2 and User3, and all 
their interactions with other threads. 

We start with (a). Let ad be such that ad\i matches Figure|7jc) for every i. Fix any dummifying function /. We must show 
that any trace originating from dummify(/, A, E, ad, /), whose log contains ad as a projected sublog, is in (pv- Additionally 
we must show that there is such a trace. There are two potential issues in mimicking the execution in ad starting from 
dummify(/, A, E, ad, f) — first, with the interaction between Userl and Notary3 and, second, with the interaction between 
Serverl and User2. For the first interaction, on line 7, ,4,(Userl) (Figure ID synchronizes with Notary3 according to I, but 
the synchronization label does not exist in ad- However, in dummify(/, ,4,, E, ad, /), the recv() on line 10 in ,4(Userl) is 
replaced with a dummy value, so the execution from dummify(/, ,4,, E,ad,/) progresses. Subsequently, the majority check 
(assertion [B]) succeeds as in /, because two of the three notaries (Notaryl and Notary2) still attest the Adversary’s key. 

A similar observation can be made about the interaction between Serverl and User2. Line 4, 4l(Serverl) (from 
Figure |3b)) synchronizes with User2 according to /, but this synchronization label does not exist in ad- However, in 
dummify(/, E, Od, /), the recv() on line 4 in ^(Serverl) is replaced with a dummy value, so the execution from 
dummify(/, ,4, E, Od, /) progresses. Subsequently, Serverl still adds permission for the Adversary. 

Next we prove that every trace starting from dummify(/, A, E, ad, /), whose log contains ad (Figure |2lc)) as a projected 
sublog, is in ipv- Fix a trace t' with log Assume I' coincides with ad- We show t' £ ipv as follows: 

1) Since the synchronization labels in V are a superset of those in ad, Serverl must execute line 10 of its program 
.4(Serverl) in t'- After this line, the access control matrix Pi contains {accti, J) for some J. 

2) When ^(Serverl) writes (x, J) to Pi at line 10, then J is the third component of a tuple obtained by decrypting a 
message received on line 7. 

3) Since the synchronization projections on /' are a superset of ad, and on ad (Serverl,?) synchronizes with 
(Adversary, 10), J must be the third component of an encrypted message sent on line 10 of ,4(Adversary). 

4) The third component of the message sent on line 10 by Adversary is exactly the term “Adversary”. (This is easy to 
see, as the term “Adversary” is hardcoded on line 9.) Hence, J ~ Adversary. 

5) This immediately implies that t' £ ipy since (accfi. Adversary) £ Pi, but Adversary ^ Userl. 

Last, we prove (b) — that no proper subsequence of ad satisfies sufficiency’. Note that ad (Figure |2tc)) contains exactly 
those actions from I (FigurelD on whose returned values the last statement of Serverl’s program (FigurelD is data or control 
dependent. Consequently, all of ad as shown is necessary to obtain the violation. 
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(a) 


^\Useri, 'Z), ^Adversary, 1)), 
(Adversary, 2), 

((Adversary, 3), (Userl, 3)), 
((Userl, 12), (Adversary, 4)), 
(Adversary, 5), 

(Adversary, 6), 

((Adversary, 7), (Serverl, 1)), 
((Serverl, 3), (Adversary, 8)), 
(Adversary, 9), 

((Adversary, 10), (Serverl, 7)), 


log(l)|L 


(Userl, l), 

((Userl, 2), (Adversary, 1)), 
((Adversary, 3), (Userl, 3)), 
(Userl, 4), 

((Userl, 5), (Notaryl, 1)), 
((Userl, 6), (Notary2,1)), 
((Userl, 7), (Notary3,1)), 
((Notaryl, 2), (Userl, 8)), 
((Notary2, 2), (Userl, 9)), 
((Notary3, 2), (Userl, 10)), 
(Userl, 11), 

((Userl, 12), (Adversary, 4)), 


(Serverl, 2), 

((Serverl, 3), (Adversary, 8)), 
((User2, 2), (Serverl, 4)), 
(Serverl, 5), 

((Serverl, 6), (User2, 3)), 
((Adversary, 10), (Serverl, 7)), 
(Serverl, 8), 

(Serverl, 9), 

(Serverl, 10), 


log(i)| 


Notaryl • 


((Userl, 5), (Notaryl, 1)), 
((Notaryl, 2), (Userl, 8)), 


log(i)| 


Notary2 • 


((Userl, 6), (Notary2,1)), 
((Notary2, 2), (Userl, 9)), 


^^g(^) iNotaryS * 

((Userl, 7), (Notary3,1)), 
((Notary3, 2), (Userl, 10)), 


((User2, 2), (Serverl, 4)), 
((Serverl, 6), (User2, 3)), 
((User2, 4), (User3,1)), 
((User3, 3), (User2, 5)), 


(Adversary, 2), 

((Adversary, 3), (Userl, 3)), 
((Userl, 12), (Adversary, 4)), 
(Adversary, 5), 

(Adversary, 6), 

((Adversary, 7), (Serverl, 1)), 
((Serverl, 3), (Adversary, 8)), 
(Adversary, 9), 

((Adversary, 10), (Serverl, 7)), 




(Userl, 1) 

((Userl, 2), (Adversary, 1)), 
((Adversary, 3), (Userl, 3)), 
(Userl, 4), 

((Userl, 5), (Notaryl, 1)), 
((Userl, 6), (Notary2,1)), 
((Userl, 7), (Notary3,1)), 
((Notaryl, 2), (Userl, 8)), 
((Notary2, 2), (Userl, 9)), 
((Notary3, 2), (Userl, 10)), 
(Userl, 11), 

((Userl, 12), (Adversary, 4)), 


(Serverl, 2), 

((Serverl, 3), (Adversary, 8)), 
((User2, 2), (Serverl, 4)), 
(Serverl, 5), 

((Serverl, 6), (User2, 3)), 
((Adversary, 10), (Serverl, 7)), 
(Serverl, 8), 

(Serverl, 9), 

(Serverl, 10), 


Notaryl • 


((Userl, 5), (Notaryl, 1)), 
((Notaryl, 2), (Userl, 8)), 


Notary2 • 


((Userl, 6), (Notary2,1)), 
((Notary2, 2), (Userl, 9)), 


NotaryS • 


((Userl, 7), (Notary3,1)), 
((Notary3, 2), (Userl, 10)), 


log(t)|user2: 



^|user2* 


((User2, 2), (Serverl, 4)), 
((Serverl, 6), (User2, 3)), 


log(t) lAdversary 



^ lAdversary 



lAdversary 


log(f) Iserverl • 



^Iserverl* 



Iserverl • 


(b) 


(c) 


mjseri, Z), ^Adversary, 1)), 
(Adversary, 2), 

((Adversary, 3), (Userl, 3)), 
((Userl, 12), (Adversary, 4)), 
(Adversary, 5), 

(Adversary, 6), 

((Adversary, 7), (Serverl, 1)), 
((Serverl, 3), (Adversary, 8)), 
(Adversary, 9), 

((Adversary, 10), (Serverl, 7)), 


Q-dluserl 

(Useri, 1) 

((Userl, 2), (Adversary, 1)), 
((Adversary, 3), (Userl, 3)) 
(Userl, 4), 

((Userl, 5), (Notaryl, 1)), 
((Userl, 6), (Notary2,1)), 

((Notaryl, 2), (Userl, 8)), 
((Notary2, 2), (Userl, 9)), 

(Userl, 11), 

((Userl, 12), (Adversary, 4)), 


(Serverl, 2) 

((Serverl, 3), (Adversary, 8)), 


((Adversary, 10), (Serverl, 7)), 
(Serverl, 8), 

(Serverl, 9), 

(Serverl, 10), 


Q'd I Notaryl • 


((Userl, 5), (Notaryl, 1)), 
((Notaryl, 2), (Userl, 8)), 


Q'(i|Notary2* 


((Userl, 6), (Notary2,1)), 
((Notary2, 2), (Userl, 9)), 


log(t)|user3: 

((User2, 4), (User3,1)), 

(User3,2), 

((User3, 3), (User2, 5)), 
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Figure 7. Left to Right: (a): log(t)|i for i £ {Adversary, Userl, Serverl, Notaryl, Notary2, NotaryS, User2, UserS}. (b): Lamport cause I for 
Theorem [ 3 ] l\i = 0 for i £ {Notary4, Server2, User4, UserS} as output by Definition 1121 (c): Actual cause for Theorem m adU = 0 for 
i £ (NotaryS, Notary4, Server2, User4, User2, UserS). ad is a projected siiblog of Lamport cause 1. 
























|Server2* 

((User4, 2), (Server2,1)), 
(Server2, 2), 

((Server2, 3), (User4, 3)), 
((User4, 8), (Server2, 4}), 
(Server2, 5), 

(Server2, 6), 

(Server2, 7), 


log(t)|userl 

(User4,1), 

((User4, 2), (Server2,1)), 
((Server2, 3), (User4, 3)), 
(Llser4,4), 

((User4, 5), (Notary4, 1)), 
((Notary4, 3), (User4, 6)), 
(User4, 7), 

((User4, 8), (Server2, 4)), 


log(t)|Notary4: 

((User4, 5), (Notary4, 1)), 
(Notary4, 2), 

((Notary4, 3), (User4, 6)), 


Figure 8. log(t)|i where i S {User4, Server2, Notary4} 

In particular, observe that if labels for Serverl (odlserveri) are not a part of a'^, then Serverl’s labels are not in 
dummify(/, ,4,, E, Od,/) and, hence, on any counterfactual trace Serverl cannot write to Pi, thus precluding a violation. 
Therefore, the sequence of labels in Odlserveri are required in the actual cause. 

By sufficiency’, for any /, the log of trace t' of dummify(/, ,4,, E, /) must contain aj, as a projected sublog. This 
means that in t', the assertion [A] of ^(Serverl) must succeed and, hence, on line 7, the correct password pwdi must be 
received by Serverl, independent of /. This immediately implies that Adversary’s action of sending that password must be 
in ad, else some dummified executions will have the wrong password sent to Serverl and the assertion [A] will fail. 

Extending this logic further, we now observe that because Adversary forwards a password received from Userl (line 4 of 
yl(Adversary)) to Serverl, the send action of Userl will be in ad (otherwise, some dummifications of line 4 of yl(Adversary) 
will result in the wrong password being sent to Serverl, a contradiction). Since Userl’s action is in ad and I' must contain ad 
as a sublog, the majority check of ,4(Userl) must also succeed. This means that at least two of {Notaryl, Notary2, NotaryS} 
must send the confirmation to Userl, else the dummification of lines 8 - 10 of AA(Userl) will cause the assertion [B] to 
fail for some /. Since we are looking for a minimal sublog therefore we only consider the send actions from two threads 
i.e. {Notaryl, Notary2}. At this point we have established that each of the labels as shown in Figure |2lc) are required in 
ad- Hence, a'j = ad- 
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